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DETAILED ACTION 

1 . This office action is in response to the amendment filed on 30 August 2007. 

2. Claims 1, 23, 24, 55, and 56 have been amended. 

3. Claims 17 and 40 have been cancelled by the Applicant. 

4. The rejection of Claims 1-2, 15, 17-18, 20-21, 23-25, 38, 40-44, 49, and 54 under 
35 U.S.C. 103(a) as being unpatentable over Eintracht et al (US Patent #6,687,878, 
filed 3/1 5/1999) in further in view of Berquist et al (US Patent #5,821 ,931 , patented 
10/13/1998) has been withdrawn as necessitated by the amendment. The rejection of 
Claims 3-6, 8, 11, 13, 26-29, 31, 34, 36, 45-48, 50-53, and 55 under 35 U.S.C. 103(a) 
as being unpatentable over Eintracht et al in further in view of Berquist et al in further 
view of Murray et al (US Patent #6,597,800, filed 9/22/1999) has been withdrawn as 
necessitated by the amendment. The rejection of Claims 12 and 35 under 35 

U.S.C. 103(a) as being unpatentable over Eintracht et al in further in view of Berquist et 
al in further view of Murray et al in further view of Takaha (US Patent #6,021 ,221 , 
patented 2/1/2000) has been withdrawn as necessitated by the amendment. The 
rejection of Claims 14 and 37 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Eintracht et al in further in view of Berquist et in further view of 
Murray et al in further view of Doyle (US Patent #6,616,701 , filed 4/3/2001 ; continuation 
of appl #09/316,496, filed 5/21/1999; provisional appl. #60/086,620, filed 5/23/1998) has 
been withdrawn as necessitated by the amendment. The rejection of Claims 19 and 42 
under 35 U.S.C. 103(a) as being unpatentable over Eintracht et al in further in view of 
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Berquist et al in further view of Balabanovic et al (US Patent #6,976,229, filed 
12/16/1999). 

5. Claims 1-6, 8, 11-15, 18-21, 23-29, 31, 34-38, and 41-56 are pending. Claims 1, 
23, 24, 55, and 56 are independent claims. 

Claim Rejections - 35 USC §112 

6. The following is a quotation of the sixth paragraph of 35 U.S.C. 112: 

An element in a claim for a combination may be expressed as a means or step for 
performing a specified function without the recital of structure, material, or acts in support 
thereof and such claim shall be construed to cover the corresponding structure, material, 
or acts described in the specification and equivalents thereof. 

7. As per independent Claim 24, Claim 24 remains rejected since the claim recites 
the limitation "providing means... and storage means..." Examiner is unsure if the 
means for is referring to any hardware since apparatus is still referring to software, pro 
se. 

Claim Rejections - 35 USC § 101 

8. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

9. Claims 24-29, 31, 34-38, 41-44, and 50-54 remains rejected under 35 U.S.C. 101 
because the claimed invention is directed to non-statutory subject matter. 

For your reference, below is a section from MPEP 2105 : 



(a) Functional Descriptive Material: "Data Structures" Representing Descriptive Material 
Per Se or Computer Programs Representing Computer Listings Per Se 
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Data structures not claimed as embodied in computer-readable media are descriptive 
material per se and are not statutory because they are not capable of causing functional 
change in the computer. See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 
(claim to a data structure per se held nonstatutory). Such claimed data structures do not 
define any structural and functional interrelationships between the data structure and other 
claimed aspects of the invention which permit the data structure's functionality to be 
realized. In contrast, a claimed computer-readable medium encoded with a data 
structure defines structural and functional interrelationships between the data structure 
and the computer software and hardware components which permit the data structure's 
functionality to be realized, and is thus statutory. 

Similarly, computer programs claimed as computer listings per se, i.e., the descriptions or 
expressions of the programs, are not physical "things." They are neither computer 
components nor statutory processes, as they are not "acts" being performed. Such 
claimed computer programs do not define any structural and functional interrelationships 
between the computer program and other claimed elements of a computer which permit 
the computer program's functionality to be realized. In contrast, a claimed computer- 
readable medium encoded with a computer program is a computer element which defines 
structural and functional interrelationships between the computer program and the rest of 
the computer which permit the computer program's functionality to be realized, and is 
thus statutory. Accordingly, it is important to distinguish claims that define descriptive 
material per se from claims that define statutory inventions. 
Computer programs are often recited as part of a claim. Office personnel should 
determine whether the computer program is being claimed as part of an otherwise 
statutory manufacture or machine. In such a case, the claim remains statutory irrespective 
of the fact that a computer program is included in the claim. The same result occurs when 
a computer program is used in a computerized process where the computer executes the 
instructions set forth in the computer program. Only when the claimed invention taken as 
a whole is directed to a mere program listing, i.e., to only its description or expression, is it 
descriptive material per se and hence nonstatutory. 

Since a computer program is merely a set of instructions capable of being executed by a 
computer, the computer program itself is not a process and Office personnel should treat 
a claim for a computer program, without the computer-readable medium needed to realize 
the computer program's functionality, as nonstatutory functional descriptive material. 
When a computer program is claimed in a process where the computer is executing the 
computer program's instructions. Office personnel should treat the claim as a process 
claim. See paragraph IV. B. 2(b), below. When a computer program is recited in 
conjunction with a physical structure, such as a computer memory. Office personnel 
should treat the claim as a product claim. 



10. Claims 24-29, 31, 34-38, 41-44, and 50-54 remain rejected under 35 U.S.C. 101 
because the claimed invention is directed to non-statutory subject matter. The claims 
appear to be claiming "software systems" i.e. systems without hardware indication, 
which is a computer program per se. Since the claims disclose computer program per 
se that is not embodied on a computer readable medium, they appear non-statutory. 
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Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 1-2, 15, 18, 20-21, 23-25, 38, 41-44, 49, and 54 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Eintracht et al (US Patent #6,687,878, filed 
3/1 5/1 999) in further in view of Berquist et al (US Patent #5,821 ,931 , patented 
10/13/1998) in further view of Schneiderman (US PGPub 2002/0054059 filed on 
10/5/2001, which is cont. of PCT/U S0 1/04963 filed 2/16/2001, which is a non- 
provisional of provisional application filed 4/17/2000, which is a non-provisional of 
provisional application filed 2/18/2000) 

As per independent Claim 1, Eintracht et al discloses a method comprising: 
• Displaying the plurality of icons, each of the icons being labelled with one or 
more of the metadata labels with which the icon was associated. (Fig 1B, 1C, 
indicator 16: Discloses a plurality of notes in a location that was 
predetermined placed being attached to various regions on the image 
(Column 7, lines 1-5) Column 15, lines 11-13, e.g., discloses text can be 
inputted which produces the note as a label, thus displayed as a metadata 
label. In addition, the note itself is the icon, while the text input is the labeling 
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aspect of the icon. Thus, Eintracht et al's note is an icon with metadata label 
capabilities of being displayed) 

• Detecting selection of at least one of the displayed plurality of icons. (Column 
15, lines 24-27: Discloses selecting a note to be dragged and dropped to 
another location on an image. The detection is inherently detected by 
positioning the cursor over the note and activating the cursor to note enabling 
it to be moved or dragged to a new location.) 

• Determining a location of a subject rendered based on a selection of the 
subject. (Column 15, lines 24-27: discloses the note being moved to a new 
different location. When the user moves the note to a new location to be 
placed, the user is inherently determining a location for the note to be placed.) 

• linking the one or more metadata labels associated with the selected icon with 
a description of the location of the selected subject within the image, and 
storing the linked one or more metadata labels and the description as an 
annotation of the image. (Eintracht et al discloses in Column 19, lines 42-67, 
notes are stored being associated with a document or image and in FIG 1 1 ; 
Column 17, lines 29-30, Eintracht et al discloses an anchor field for each note 
in a database that stores the coordinates of the of the note in the document 
thus linking the note to the document and disclosing the precise location of 
the note in the document.) 

Eintracht et al fails to specifically disclose when determining a location on a 
image of one or more metadata labels associated with the selected icon being related to 
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said selected subject. However, Eintracht et al discloses that text can be entered into 
the metadata area (Column 15, lines 11-13). It would have been obvious to one of 
ordinary skill in the art at the time of Applicant's invention that text inputted into a note 
that it would contain related subject matter to the image the note would be placed on, 
then place the note with the information onto the subject in the image would have 
provided the benefit of a user using the note annotation to know the details regarding 
the subject matter within an image or document. 

In addition, Eintracht et al fails to specifically disclose displaying the image 
adjacent to said plurality of icons. However, Berquist et al discloses in FIG 4, a number 
of notes being displaying adjacent to an application program executing window 
containing a document. In addition, FIG 6 discloses a note placed adjacent to a window 
containing a document before being dragged onto the document. While Berquist et al 
discloses an embodiment of notes being adjacent to document, Berquist et al discloses 
a note may be attached to a graphic or a video frame. It was well-known in the art at the 
time of Applicant's invention document have the ability to contain images, or the 
application program running had the ability to only show an image, thus therefore it 
would have been obvious to one of ordinary skill in the art at the time of Applicant's 
invention that a note could be displayed adjacent to the image within a document or 
within an application program since Berquist et al would have provided the benefit to 
Eintracht et al in which preventing objects on the screen from overlapping, and being 
able to determine a location prior in moving the object onto the desired location. 
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However, Eintracht and Berquist fail to specifically disclose extracting a plurality 
of metadata labels from an extracting a plurality of metadata labels from an existing 
database of metadata labels to form a list of metadata labels; said metadata labels are 
generated prior to having knowledge of the content of the image. However, 
Schneiderman discloses list of metadata labels, i.e. names of people that appear in a 
selectable menu list wherein the list of names have been previously entered into the 
database. (FIG 3, Paragraph 0011, 0015, 0049) Thus, the names were extracted from 
the database and displayed into a selectable list before having knowledge of the content 
of the image appearing next to the list. 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have modified Eintracht and Berquist with Schneiderman ability 
to display a list of names labels that were extracted from a database since it would have 
provided the benefit of addresses known problems of annotating commercial and/or 
personal electronic images, by providing software that permits users to easily 
accomplish such annotation, through the drag-and-drop of annotations from a 
predefined list. 

As per dependent Claim 2, Claim 2 recites similar limitations as in Claim 1 and is 
rejected under similar rationale. Eintracht et al discloses a method: 

• wherein the selection of said subject is detected by dragging the selected icon 
to the image and dropping the dragged icon on the subject of the image. 
(Column 15, line 24-27: Discloses selecting a note to be dragged and 
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dropped to another location on the image. The detection is inherently 
detected by positioning the cursor over the note and activating the cursor to 
note enabling it move dragged to a new location. When the user moves the 
note to either part of the image and placed, the note is anchored to that 
coordinate (Column 7, lines 62-64) therefore the subject selected is detected.) 



As per dependent Claim 15, Claim 15 recites similar limitations as in Claim 1 and 
is rejected under similar rationale. Furthermore, Eintracht et al discloses: 

• wherein one or more metadata labels are stored as the annotation of the 
subject and are displayed upon selecting the subject in the image. (Column 7, 
lines 13-17, discloses when a note is selected on the note list, its counterpart 
icon in the document window frame is highlighted) 



As per dependent Claim 18, Eintracht et al discloses a method: 
• wherein said storing step included storing the one or more metadata labels as 
the annotation of the subject of the image by using a tag indicating an 
association with the image (Column 3 Lines 13-36 i.e. the document file for 
storing one or more documents, a notes database located on the server, the 
notes database for storing one or more notes, each note or tag associated 
with a particular document or subject, one or more notes clients coupled to a 
network, each notes client operative to locally display a representation of a 
document remotely stored on the server in the document file, the notes client 
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adapted to permit a user to annotate the document with one or more notes, 
the notes client operative to simultaneously display the one or more notes 
associated with the document over the displayed document such that the 
document is viewable along with the one or more notes 

As per dependent Claim 20, Eintracht et al discloses a method further 

comprising: 

• the step of e-mailing at least the image to at least one e-mail address based 
on the one or more metadata labels associated with the image. (Column 22 
Lines 33-37 i.e. emails to client or user) 

As per dependent Claim 21 , Eintracht et al discloses a method further 
comprising: 

• the step of replacing a default icon by the selected icon based on the subject 
of the image. (Column 15, lines 10-37,discloses the ability to view, create, 
modify, or delete notes. Therefore, one can delete the icon located on the 
selected subject, and either create a new note or move a note to a new 
location onto that subject). 

As per independent Claim 23, Claim 23 recites a "computer readable medium..." 
for performing the method of Claim 1, and therefore is similarly reject under Eintracht et 
al and Berquist et al. 
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As per independent Claim 24, Claim 24 recites an apparatus for performing the 
method of Claim 1, and therefore is similarly rejected under rationale. Furthermore, 
Eintracht et al disclose a display, selection and storage means (Column 28, lines 1-27) 

As per dependent Claim 25, Claim 25 recites similar limitations as in Claim 2 and 
is similar rejected under rationale. 

As per dependent Claim 38, Claim 38 recites similar limitations as in Claim 15 
and is similar rejected under rationale. 

As per dependent Claim 39, Claim 39 recites similar limitations as in Claim 16 
and is similar rejected under rationale. 

As per dependent Claim 41, Claim 41 recites similar limitations as in Claim 18 
and is similar rejected under rationale. 

As per dependent Claim 43, Claim 43 recites similar limitations as in Claim 20 
and is similar rejected under rationale. 

As per dependent Claim 44, Claim 44 recites similar limitations as in Claim 21 
and is similar rejected under rationale. 

As per dependent Claim 49, Eintracht et al discloses a method: 
• the linked one or more metadata labels and the descriptions are stored as an 
annotation of the subject of the image. (Eintracht et al discloses in Column 
19, lines 42-67, notes are stored being associated with a document or image 
and in FIG 11; Column 17, lines 29-30, Eintracht et al discloses an anchor 
field for each note in a database that stores the coordinates of the of the note 
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in the document thus linking the note to the document and disclosing the 
precise location of the note in the document.) 

As per dependent Claim 54, Claim 54 recites similar limitations as in Claim 49 
and is similar rejected under rationale. 

As per independent claim 56, Claim 56 recites similar limitations as in Claim 1 
and is similar rejected under rationale. Furthermore, Shneiderman discloses displaying 
a representation of each of the metadata labels in the list. (FIG 3) 

13. Claims 3-6, 8, 11, 13, 26-29, 31, 34, 36, 45-48, 50-53, and 55 are rejected under 
35 U.S.C. 103(a) as being unpatentable over Eintracht et al (US Patent #6,687,878) in 
further in view of Berquist et al (US Patent #5,821,931) in further view of Schneiderman 
(US PGPub 2002/0054059) in further view of Murray et al (US Patent #6,597,800, filed 
9/22/1999) 

As per dependent Claims 3 and 4, Eintracht et al, Berquist et al, and 
Schneiderman fail to specifically disclose the bounded region is formed based on an 
analysis of pixels of the image and the analysis of the colour information of the pixels of 
the image. However, Murray et al discloses a process in which an image is captured by 
a camera utilizing a two dimensional array of light intensity sensitive pixels that carries 
out the processing on the image data to separate and identify objects from the 
background appearing in the image. (Column 1, lines 15-30) Then, Murray et al 
discloses the bounding box encloses segmented pixels forming homogeneous region of 
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objects of interest. (Column 2, lines 63-65; Column 4, lines 27-30) Therefore, when 
objects are being identified, being bounded with boxes and separated based on the 
interest, pixels are being analyzed to determined which pixels are the same within the 
region to be separate from the different region of pixels based on the color of the pixel 
or the difference of visible light shown. 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, and 
Schneiderman's method with Murray et al's method since Murray et al's method since 
Murray et al's method would have provided the benefit of an automatic recognition of a 
target object that improves the accuracy of target object recognition and identification. 

As per dependent Claims 5, 13, 48, and 53, Eintracht et al, Berquist et al, and 
Schneiderman fail to specifically disclose the bounded region is of a predetermined size 
or determined automatically, and the size of the bounded region is determined based on 
the analysis. However, Murray et al discloses the bounding box just encloses the 
segmented pixels forming a primary homogenous region. (Column 2, lines 63-65; 
Column 4, lines 27-30) Therefore, only bounding related pixels, the box is determined 
automatically and is predetermined based on the number of related pixels. In addition, 
when objects are being identified, being bounded with boxes and separated based on 
the interest, pixels are being analyzed to determined which pixels are the same within 
the region to be separate from the different region of pixels. Thus, the size is being 
determined to form a homogenous region. 
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It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, and 
Schneiderman's method with Murray et al's method since it provided the benefit of only 
bounding a region of related pixels of interest from regions of no interest. 

As per dependent Claim 6, Eintracht et al, Berquist et al, and Schneiderman fail 
to specifically disclose the step of forming a bounded region within the image about the 
location at which the subject is rendered in said image, the bounded region being 
configured substantially surround the subject. However, Murray et al discloses a 
bounding box is the rectangle which encloses the segmented pixels forming a primary 
homogenous region which depicts objects of interest (Column 2, lines 63-65; Column 4, 
lines 27-30) 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, and 
Schneiderman's method with Murray et al's method since Murray et al's method since 
Murray et al's method would have provided the benefit of an automatic recognition of 
subjects to identify the presence of people. 

As per dependent Claim 8, Eintracht et al discloses dragging an icon. Column 15, 
line 24-27) However, Eintracht et al, Berquist et al, and Schneiderman fails to 
specifically disclose a bounded region under the dragged icon is emphasized. On the 
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other hand, Murray et al discloses a bounding box is a rectangle enclosing a region, 
thus therefore the bounded region being emphasized. 

It would have been oblivious to one of ordinary skill in the art the time of 
Applicant's invention to have combined have combined Eintracht et al, Berquist et al, 
and Schneiderman's method with Murray et al's method of bounding box since it 
provide the benefit of a clear identification of an object that's being identified in an 
image. (Column 2, lines 63-65; Column 4, lines 27-30) 

As per dependent Claim 1 1 , Eintracht et al, Berquist et al, and Schneiderman fail 
to specifically disclose the step of extracting a part of the image based on the bounded 
region. However, Murray et al discloses an image being divided into one or more 
primary homogenous regions and extracting the data from the image regarding the 
primary regions. (Column 2, lines 63-67) 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, and 
Schneiderman's method with Murray et al's method since it provided the benefit of 
extracting individual complete objects for further image processing. 

As per dependent Claim 26, Claim 26 recites similar limitations as in Claim 3 and 
is similarly rejected under rationale. 

As per dependent Claim 27, Claim 27 recites similar limitations as in Claim 4 and 
is similarly rejected under rationale. 
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As per dependent Claim 28, Claim 28 recites similar limitations as in Claim 5 and 
is similarly rejected under rationale. 

As per dependent Claim 29, Claim 29 recites similar limitations as in Claim 6 and 
is similarly rejected under rationale. 

As per dependent Claim 31, Claim 31 recites similar limitations as in Claim 8 and 
is similarly rejected under rationale. 

As per dependent Claim 34, Claim 34 recites similar limitations as in Claim 1 1 is 
similarly rejected under rationale. 

As per dependent Claim 36, Claim 36 recites similar limitations as in Claim 13 is 
similarly rejected under rationale. 

As per dependent Claims 45 and 46, Eintracht et al, Berquist et al, and 
Schneiderman to specifically disclose the description includes a location and the size of 
the bounded region within the image. However, Murray et al discloses a bounding box 
encloses segmented pixels forming a primary homogenous region wherein the 
information is transmitted to an extraction device. Since a bounding box is formed 
around the pixels, the size is by the pixels within the box and a location is found by 
where the box is located. Thus, it would have been obvious to one of ordinary skill in the 
art at the time of Applicant's invention to have combined Eintracht et al, Berquist et al, 
and Schneiderman's method with Murray et al's method of a bounding box since the 
bounding box data would have provided the user information on the location of a object 
of interest and the size of the object within an image. 
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As per dependent Claim 47, Eintracht et al discloses dragging an icon. Column 
15, line 24-27) However, Eintracht et al, Berquist et al, and Schneiderman fail to 
specifically disclose a bounded region is formed at a location at which the selected icon 
is dropped on the image. On the other hand, Murray et al discloses a bounding box is a 
rectangle enclosing a region of related pixels of an object of interest. (Column 2, lines 
63-65; Column 4, lines 27-30) 

It would have been oblivious to one of ordinary skill in the art the time of 
Applicant's invention to have combined have combined Eintracht et al, Berquist et al, 
and Schneiderman's method with Murray et al's method of bounding box since it 
provide the benefit of a clear identification of an object that's being identified in an 
image. 

As per dependent Claims 50-51, Claim 50-51 recites similar limitations as in 
Claim 45-46 and is similarly rejected under rationale 

As per dependent Claims 52, Claim 52 recites similar limitations as in Claim 47 
and is similar rejected under rationale. 

As per independent Claim 55, Claim 55 recites similar limitations as in Claim 1 
and 6 combined and therefore is similar rejected under rationale. 

14. Claims 12 and 35 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eintracht et al (US Patent #6,687,878, filed 3/15/1999) in further in view of Berquist 
etal (US Patent #5,821,931, patented 10/13/1998) in further view of Schneiderman (US 
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PGPub 2002/0054059) in further view of Murray et al (US Patent #6,597,800, filed 
9/22/1999) in further view of Takaha (US Patent #6,021,221, patented 2/1/2000). 

As per dependent Claim 12, Eintracht et al, Berquist et al, Schneiderman and 
Murray et al fail to specifically disclose displaying the extracted part of the image. 
However, Takaha discloses displaying the extracted image of the pixels that were 
extracted from the original image (Column 6, lines 2-5; Column 15, line 66 - Column 16, 
2) 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, Schneiderman 
and Murray et al's methods with Takaha's method since Takaha's method would have 
provided the benefit of only showing areas of interest within an image after extracting. 

As per dependent Claim 35, Claim 35 recites similar limitations as in Claim 12 
and is similar rejected under rationale. 

15. Claims 14 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eintracht et al (US Patent #6,687,878, filed 3/15/1999) in further in view of Berquist 
etal (US Patent #5,821,931, patented 10/13/1998) in further view of Schneiderman (US 
PGPub 2002/0054059) in further view of Murray et al (US Patent #6,597,800, filed 
9/22/1999) in further view of Doyle (US Patent #6,616,701 , filed 4/3/2001 ; continuation 
of appl #09/316,496, filed 5/21/1999; provisional appl. #60/086,620, filed 5/23/1998). 

As per dependent Claim 14, Eintracht et al, Berquist et al, Schneiderman, and 
Murray et al fail to specifically disclose that the size of the bounded region is 
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changeable by the user. However, Doyle discloses objects in the image data are 
interactively outlined in which the user is present. (Column 3, lines 23-25) Since the 
process of outlining can be done interactively, the user is defining and/or changing the 
size of the bounded region. 

It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, Schneiderman, 
and Murray et al's methods with Doyle's method since Doyle's method would have 
provided the method allowing objects within in a single multidimensional dataset to be 
mapped. 

As per dependent Claim 37, Claim 37 recites similar limitations as in Claim 14 
and is similar rejected under rationale. 

16. Claims 19 and 42 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eintracht et al (US Patent #6,687,878, filed 3/15/1999) in further in view of Berquist 
et al (US Patent #5,821 ,931 , patented 10/13/1998) in further view of Schneiderman (US 
PGPub 2002/0054059) in further view of Balabanovic et al (US Patent #6,976,229, filed 
12/16/1999). 

As per dependent Claim 19, Eintracht et al, Berquist et al, and Schneiderman fail 
to specifically disclose the one or more predetermined metadata labels associated with 
the subject of the image are stored in an XML file. However, Balabanovic et al discloses 
metadata regarding information to an image being stored in an XML format/file. (FIG 
5(a,b)-6; Column 10, lines 1-3, 21- 22, 40-41) 
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It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to have combined Eintracht et al, Berquist et al, and 
Schneiderman's method with Balabanovic et al's method of using metadata in XML 
format since it would have provided the benefit of flexibility to for the file to be easily 
translated into other formats to viewed by other on different devices. 

As per dependent Claim 42, Claim 42 recites similar limitations as in Claim 13 
and is similar rejected under rationale. 

Response to Arguments 

17. Applicant's arguments with respect to claims 1-6, 8, 11-15, 18-21 , 23-29, 31 , 34- 
38, 41-56 have been considered but are moot in view of the new ground(s) of rejection. 

Arguments address regarding of the new limitations of Claims 1, 23, 24, 55, and 
56 brought forth in extracting a plurality of metadata labels from an extracting a plurality 
of metadata labels from an existing database of metadata labels to form a list of 
metadata labels; said metadata labels are generated prior to having knowledge of the 
content of the image has been viewed the new ground of rejection of 35 USC 103(a) 
under new references using Schneiderman. 

18. Applicant's arguments filed 30 August 2007 have been fully considered but they 
are not persuasive. 

19. On page 14, in regards Claims 24-29, 31, 34-38, 41-44, and 50-54, Applicant 
argues that the claims contains statutory subject matter under 35 USC 101 wherein the 
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Applicant states the claims "also seen to correspond with hardware." However, the 
Examiner disagrees. 

Claims 24-29, 31, 34-38, 41-44, and 50-54 disclose a an apparatus for 
annotating an image wherein the claims themselves as written fail to disclose any form 
of hardware indication since the claims do not disclose the use of any hardware. 
Therefore, the claims, themselves , lack the necessary physical articles or objects to 
constitute a machine or a manufacture within the meaning of 35 USC 101. They are 
clearly not a series of steps or acts to be a process nor are they a combination of 
chemical compounds to be a composition of matter. As such, they fail to fall within a 
statutory. They are, at best, functional descriptive material per se. Thus, in regards to 
claims 24-29, 31, 34-38, 40-44, and 50-54, the claims, as written , appear to be 
claiming "software systems" i.e. systems without hardware indication, which is computer 
program per se. The claims as written do not recite any hardware indication. 

20. In regards to Applicant's arguments of the 35 USC 1 12, sixth paragraph, rejection 
to Claim 1 , Applicant argues the claim provides a clear means plus function step within 
the bounds of an apparatus. However, the Examiner disagrees. Since Claim 1 still 
remains rejected under 35 USC 101 as being view a "software system", the Examiner is 
still unsure the means for is referring to any hardware since apparatus is still referring to 
software, pro se. 
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21 . On page 7, Applicant argues that the IDS submitted on 13 40 March 2006 is in 
compliance with 37 CFR 1.98(a)(1) stating the EPO Search Report is a source of the 
references and does not necessarily have to be listed. However, the Examiner 
disagrees. 

According to 37 CFR 1.98(a)(1), 37 CFR 1.98(a)(1) clearly states A list of all 
patents, publications, applications, or other information submitted for consideration by 
the Office. The EPO Search Report is considered a publication since it released to the 
public by the EPO for the corresponding European application. Therefore, then the EPO 
Search Report is considered a publication, it requires to be listed in the IDS to comply 
with 37 CFR 1.98(a)(1). 

Conclusion 

22. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David Faber whose telephone number is 571-272-2751. 
The examiner can normally be reached on M-F from 8am to 430pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong, can be reached on 571-272-4124. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



David Faber 
Patient Examiner 
AU 2178 
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